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(57) ABSTRACT 

A video compression system comprises a control computer 
52; a plurality of encoders 54, 56, 58; a plurality of encoder 
buffers 60, 62, 64; a multiplexor 66; a data channel 70; a 
demultiplexer 80; a decoder buffer 82; a decoder 84; and 
display 86. The control computer controls the encoders and 
the multiplexor 66 to avoid overflows and underflows of 
data provided to the decoder buffer 82. 
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METHOD AND APPARATUS FOR 
EFFECTING SEAMLESS DATA RATE 
CHANGES IN A VIDEO COMPRESSION 
SYSTEM 

FIELD OF THE INVENTION 

The present invention relates generally to the fields of 
digital television and video compression, and more particu- 
larly relates to methods and apparatus for effecting seamless 
rate changes in a video compression system employing 
buffers. 

BACKGROUND OF THE INVENTION 

The background of the present invention is described 
herein in the context of pay television systems, such as cable 
television and direct broadcast satellite (DBS) systems. 
However, the invention is by no means limited thereto 
except as may be expressly set forth in the accompanying 
claims. 

In the pay television industry, programmers produce pro- 
grams for distribution to various remote locations. A "pro- 
gram" may consist of video, audio, and/or other related 
services, such as closed-captioning and teletext services. A 
programmer supplies these services, e.g., via satellite, to 
individual subscribers and/or cable television operators. 
Typically, many separately encoded, compressed programs 
are multiplexed together and sent over a data channel to 
cable television operators. The cable operators receive the 
multiplex and select the programs/services they wish to 
distribute to their subscribers. The selected programs are 
then transmitted to the individual subscribers via a coaxial 
cable distribution network. 

In the past, pay television systems have operated in the 
analog domain. Recently, however, the pay television indus- 
try has begun to move toward all digital systems wherein, 
prior to transmission, the analog program signals are con- 
verted to digital signals. Digital signal transmission offers 
the advantage that digital data can be processed at both the 
transmission and reception ends to improve picture quality. 
In addition, digital data compression techniques can achieve 
high signal compression ratios. Digital compression allows 
a larger number of individual services to be transmitted 
within a fixed bandwidth. Bandwidth limitations are 
imposed by both satellite transponders and coaxial cable 
distribution networks, and therefore digital compression is 
extremely advantageous. 

FIG. 1 depicts an exemplary application of a vector 
quantization image compression system wherein image data 
is communicated from a point of origin 12 to receiving 
locations 14, 16. (It should be noted that the invention is not 
limited to use in a vector quantization system, or any other 
particular type of encoding system. For example, the inven- 
tion may be employed in a system using discrete cosine 
transform (DCT) encoding, such as an MPEG system.) The 
point of origin 12 might include a source 20 of program 
material that supplies movies and the like in analog form to 
an encoder 22 for digitization and data compression by 
vector quantization. Compressed digital data is transmitted 
to a satellite 18 via transmitter 24 for reception by a plurality 
of earth stations 14, 16. The earth stations 14, 16 may be the 
head-end of a cable television distribution system of the type 
which receives signals from the satellite 18 and distributes 
them to a plurality of subscribers via coaxial cable or optical 
fiber. Alternatively, one or more of the earth stations may be 
DBS (direct broadcast satellite) subscribers who receive 
signals directly from the satellite 18. The cable head-end 
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installation 14 may receive the data transmitted by the 
station 12 via a receiver 26 and then employ an on-site 
decoder 28 for decompressing the received data and con- 
verting the same back to analog form. Another on-site 

5 apparatus 30 may convert the analog data to conventional 
NTSC signals for transmission over cable to subscribers in 
conventional form. Thus, in the case of cable head-end 
installation 14, the cable head-end operator distributes ana- 
log NTSC cable television signals to subscribers in conven- 

10 tional form. Further background on digital television can be 
found in U.S. patent application Sen No. 968,846, Oct. 30, 
1992, titled System and Method for Transmitting a Plurality 
of Digital Services, and U.S. Patent No. __ (application Ser. 
No. 794,516, issued Dec. 15, 1992), titled Image Compres- 

1 5 sion Method and Apparatus Employing Distortion A daptive 
Tree Search Vector Quantization. 

A problem with the prior art is caused by the fact that, 
when the program provider decides to change the data rate 
at which a given program is encoded and provided to cable 

20 operators, there is often a noticeable interruption in the 
picture received by the viewers that are watching that 
program. One reason for effecting such a rate change would 
be to increase the amount of data received by the cable 
operators and to thereby improve the quality of the televi- 

25 sion pictures received by the viewers. Typically, the program 
multiplex is provided to the data channel at a fixed rate, 
although the individual programs composing the multiplex 
are compressed with a variable rate coding scheme. This 
means that buffers are required at the encoder(s) and decoder 

30 (s). If the encoding rate R ( - of one program is changed in a 
system with a fixed rate data channel, the encoding rates of 
all other programs must be adjusted such that the total data 
rate R is fixed (where R equals the sum of the data rates of 
the individual program channels). As programs are encoded 

35 by a program provider prior to being sent to a cable operator, 
the encoding rate for each program must be controlled to 
avoid decoder buffer overflows and underflows, which, if 
allowed to occur, would produce a noticeable interruption in 
the viewers* television pictures. 

40 Accordingly, a primary goal of the present invention is to 
provide methods and apparatus whereby the encoding rate is 
adjusted automatically to avoid any overflows or underflows 
in the viewers' decoder buffers. 

45 SUMMARY OF THE INVENTION 

The present invention provides methods and apparatus 
especially suited for video compression systems (e.g., pay 
television systems) comprising at least one encoder for 

50 compressing program data, an encoder buffer, and one or 
more decoders connected to associated decoder buffers. An 
example of such a system is described below in connection 
with a detailed description of one prefer red embodiment of 
the invention. The present invention will most often be 

55 employed by program providers th at send compressed 
digital program data to cable operators. The cable operators 
select the particular programs to provide to their viewers. 
The present invention allows the program provider to 
change the rate at which program data is encoded and 

60 transmitted to cable operators such that the rate change is 
seamless, i.e., such that the decoder buffers neither overflow 
nor underflow. 

A method for operating a video compression system in 
accordance with the present invention comprises the steps of 

65 (a) changing the rale at which an encoder provides data to an 
encoder buffer from a first rate R to a second rate R'; and (b) 
maintaining the encoder buffer occupancy b^t) such that the 



04/27/2004, EAST version: 1.4.1 



US 6,188,729 Bl 



following conditions are maintained: (1) prior to time 
T 0 -T„, max{0, RT rf -B,} ib,(t)imin{B„ KT d }; and (2) 
after time T 0 , max{0, RT rf -B rf }^b tf (t) £min{B 3 , RT rf }; 
and (3) during the time T 0 -T^tiT 0 , max{0, RT 0 -Rt+ 
R't+RT^-RTo-BrfJ^b^O^miniB^ RT 0 -Rt + R't + 5 
RT rf RT 0 }; wherein t represents time, T 0 represents the 
time at which the data rate changes from R to R\ B d 
represents the maximum capacity of the decoder buffer, and 
r X d represents the delay caused by the encoder and decoder 
buffers. The value of T d will typically be determined by the io 
startup strategy used by the system; it is the delay between 
the time a compressed picture enters the decoder and the 
time the picture is removed from the decoder buffer for 
decompression and display. 

A presently preferred implementation of the present 35 
invention further comprises the steps of multiplexing the 
data from the encoder buffer with data from at least one 
other encoder buffer, and transmitting a multiplex compris- 
ing the multiplexed data over a data channel to the decoder 
buffer. The invention may be used in connection with vector 20 
quantization and DCT encoding schemes. Other schemes 
may be used as well. 

The present invention also provides means for carrying 
out the inventive method summarized above. 

25 

In addition, the present invention provides a system for 
providing a plurality of programs to a cable operator or 
individual subscriber having means for selecting a desired 
program for viewing, a decoder and a decoder buffer pro- 
viding the selected program to the decoder. The system 3Q 
comprises first and second encoders (although more than 
two encoders may be employed) each receiving program 
data; first and second encoder buffers respectively coupled 
to the first and second encoders; a multiplexor receiving data 
from the first and second encoders and outputting a multi- 35 
plex to a data channel for reception by the cable operator or 
subscriber; and a control computer coupled to the first and 
second encoders and to the multiplexor, the controller com- 
prising program means for changing the rates at which the 
encoders provide data to their encoder buffers and the 4Q 
encoder buffers provide data to the multiplexor. According 
to the invention, the control computer controls the occupan- 
cies of the encoder buffers such that the decoder buffer 
neither overflows nor underflows when the rate at which 
either encoder provides data to its buffer is changed from a 45 
first rate R to a second rate R\ 

Other features of the present invention are described 
below. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a block diagram of a pay television system. 

FIG. 2 is a block diagram of one embodiment of a video 
compression system in accordance with the present inven- 
tion. 

FIG. 3 depicts the constraints the system of FIG. 2 places 
on the encoder buffer occupancy as the compression rate is 
increased from 4 Mbps to 6 Mbps. 

FIG. 4 depicts the constraints placed on the encoder buffer 
occupancy as the compression rate is decreased from 8 60 
Mbps to 4 Mbps and to 1.2 Mbps. 

DETAILED DESCRIPTION OF PREFERRED 
EMBODIMENTS 

FIG. 2 is a block diagram of a video compression system 65 
in accordance with the present invention. The system 
includes a control computer 52; plurality of encoders 54, 56, 



58; a plurality of encoder buffers 60, 62, 64; a multiplexor 
66; a data channel 70; a demultiplexer 80; a decoder buffer 
82; a decoder 84; and display 86. Of course, although only 
one is shown, there would normally be many additional 
multiplexor-decoder buffer-decoder-display combinations. 
In addition, the decoder 84 and display 86 may be contained 
in a television set, while the demultiplexer 80 and decoder 
buffer 82 may be located at the site of the cable operator or 
in a separate set top unit. As indicated in FIG. 2, the 
multiplexor 66 transmits data at a prescribed rate R to the 
data channel 70 and the demultiplexer 80 provides data to 
the decoder buffer 82 at the rate R y , where the index i (i=l, 
2, . . . N) represents the selected channel. The present 
invention relates to the manner in which the control com- 
puter controls the encoders and multiplexor 66 to avoid 
overflows and underflows of data provided to the decoder 
buffer 82. Such control is particularly important when the 
rates R 1 , R 2 , . . . R^ at which the encoders provide data to 
their buffers (which are the same as the rates at which the 
buffers provide data to the multiplexor 66) are changed. 
Accordingly, this specification describes in detail the man- 
ner in which such overflows and underflows may be 
avoided. The other components are known in the art and thus 
are not described in detail herein. 

Assume a variable-rate compression scheme and a 
variable-rate data channel are employed. In a representative 
case, the encoder buffer occupancy is given by: 

ft,(0) = 0 



V(0 = J%(r)rf r, for 0 & t s T e 



In the above equations, e(x) is the instantaneous bit rate 
out of the encoder, where x is a time variable used in the 
integration; T e is the time at which the multiplexor 66 starts 
reading data out of the buffer; B e is the maximum capacity 
of the encoder buffer, in bits; and R(x) is the rate at which 
the multiplexor 66 reads data from the buffer for output to 
the data channel 70 (this is preferably a constant). Similarly, 
the decoder buffer occupancy equations are: 

b d (0) = 0, for / £ T t 
b d {t) = ^R(T)dT t for T e zt£T d 



MO = Y + f T R{T)dT ~ £ Td<;{T)dT < for ' * r «* 



In the decoder equations, the terms have meanings similar 
to their counterparts in the encoder equations. The decoder 
equations are based on the premise that the first bits removed 
from the buffer at the decoder are the first bits added to the 
buffer at the encoder, and that the bits are removed at the 
same rate as they are added, taking into account the delay. 

The present inventor has determined that the sum of the 
encoder and decoder buffer occupancies is a constant when 
the encoder is operating at a fixed rate and the effects of 
encoder and decoder startup have disappeared, i.e., when the 
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encoder-decoder pair is in its "steady-state" mode. Of -continued 

course, for this condition to be true, one must take into rt 

account the delay through the system. That is, if we take the bt (r " Td) * H{T)dT " ° d 

decoder buffer occupancy at some arbitrary time i Qt we must 

take the encoder buffer occupancy at t 0 -T rf , where 7 d is the 5 

delay experienced by the data as it travels from the encoder The top constraint prevents underflows, i.e., the situation 

buffer to the decoder buffer. Accordingly, we are interested where b </0) fa less man or e( l ual t0 zero > and me lower 

in the sum of the decoder and encoder occupancies, bj(t)+ constraint P revents overflows, wherein b/t) is greater than 

b ff (t-T rf ), at t^T rf and t-T rf ^T„ or &T d +T e . 30 ° r eqUal t0 Brf * 

By a change in variable (t«t-T rf ) , the above inequalities 
It is apparent from the above equations that this sum is can be rewritten as 
given by: 



R{T)dT 

B, r-U r -r d MO* J Rir)dr-B d 

y + J e(r)rfr- j R{z)dr Jl 



= y + y- f ' e(r)dT + 2 o ' n a constant bil rale operation, i.e., where R(x) is a 

0 constant R, we have 



R{T)dT 



f R{T)dT- f 

B d r r- T d 

1 JT d JT e 



b^RT, 



25 



Bd t- f ' Td R(T)dT+ P R(r)dT- 
Jt, Jt-T, 



We also have the practical constraint 

C d R{r)dT- f 7<s R{r)dr Therefore, the encoder buffer occupancy constraint may 

** T * J** 30 be expressed as 

max (0, RT^Bj^b^O^min (B et RTd) 

From these equations, it follows that the sum of the Assume now ^ lhe data fate cfa ffom R tQ Rt at 
encoder and decoder buffer occupancies can be expressed as t j me t s j p or t <x -T we have 



1 Ji-T d JT e 



maxCORT^B^b^min (B e) RT rf ) 

For t^T^, we have 

t -T d R[T)dT 40 max(O f RT^B rf )£b,(t)^min(B,, RT rf ) 

For T n -T.^[^J n , we have 



When R(x) is a constant R, this equation becomes: bj(l)+ r Tq Td v 

b c (t-T^)=RT rf , where RT^ is a constant. The decoder buffer *r« * miJ fl„ J ffrfr + R>dA 

occupancy can thus be expressed in terms of the encoder 45 ' T ° 

buffer occupancy and the instantaneous rate at which data is b (/J a ( Q f T ° RdT + C* T<S # j T _ # ] 

read from the encoder R(x): ' \ ' Ji Jz-q j 

MO - | R(T)dr-b e (t - for t*T d + T e 50 These inequalities may be reduced to the following: 

b^t) S minCB,., RT 0 - Rt+R't+RT^-RTo) 
b,(t)&max(0, RT 0 - R t+R' t+RTj- R T 0 - B rf ) 

To prevent overflows and underflows, i.e., to ensure 
seamless rate changes, we require 55 The encoder buffer occupancy constraint expressed above 

is graphically represented by FIGS. 3 and 4. In the example 
n of FIG. 3, the data rate is changed from R=4 Mbps to R'=6 

° * i-T d RiT)dT ' M " Td) * Bd Mb P s > wilh T ^ scc - For t ^T 0 , we have RT>B^1 Mb, 

or RT rf =3 Mb. Therefore, the encoder buffer occupancy 
60 must be controlled to lie between the broken lines to ensure 
From the above inequality, the encoder buffer occupancy ^6 rate change is seamless. 

should be constrained as follows: . FIG D \ S7 P i? m ^ 'I? J* ,nf ?? 

from R=8 Mbps to R'«4 Mbps and R'=1.2 Mbps, with 

B e =B rf =2Mb and T^A sec. For the change to R'«4 Mbps, 
b<U-T d )z f R(r)dT 65 we have R r I>2 Mb, RT rf -B rf =0, R r I>l Mb, and R r l> 

J '~ T d B^-- 1 Mb. For the change to R-1.2 Mbps, we have 

RT>300,000 bits, and R'T^B^-1.7 Mb. 
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These examples show that the buffer size employed 
should be a function of the data rate. Smaller buffers are 
required at low rates. This allows the video delay through the 
system to be held constant at all data rates. 

It should be understood that the scope of protection of the 
following claims is not intended to be limited to the specific 
embodiment described above, except where the claims are 
expressly so limited. For example, the invention may be 
applied in non-digital television systems wherein data is sent 
from an encoder to a decoder via buffers and where it is 
important to maintain the decoder buffer occupancy within 10 
predetermined limits. Such applications will become appar- 
ent to those skilled in the art after reading this specification. 

I claim: 

1. A method for operating a video compression system 
comprising at least one encoder providing data to an encoder 
buffer and a decoder taking data from a decoder buffer, 
comprising the steps of: 

(a) changing the rate at which the encoder provides data 
to the encoder buffer from a first rate R to a second rate 
R'; and 

20 

(b) maintaining an encoder buffer occupancy b tf (t) such 
that: (1) prior to time T 0 -T rf , max{0, RT^Bjib^t) 
^min{B c , RT^}; and (2) after time T 0 , max{0, RT rf - 
B rf }^b,(t)^min{B,, RTj; and (3) during the time 
T 0 -T d ^T 0t max{0, RT 0 -Rt+R't+R'T rf -RT 0 - 2 , 
Bj^b^O^minfB,, Rl^-Rt+R'l+R'T^-RTo}; 
wherein t represents time, T a represents the time at 
which the data rate changes from R to R', B^ represents 
the maximum capacity of the decoder buffer, B c rep- 
resents the maximum capacity of the encoder buffer, 3Q 
and T d represents the delay caused by the encoder and 
decoder buffers. 

2. The method recited in claim 1, further comprising the 
steps of multiplexing the data from the encoder buffer with 
data from at least one other encoder buffer, and transmitting 
a multiplex comprising the multiplexed data over a data 
channel to the decoder buffer. 

3. The method recited in claim 1, further comprising the 
step of encoding the data with the encoder and providing the 
encoded data to the encoder buffer. 

4. The method recited in claim 3, wherein the encoding 40 
comprises compressing the data with a vector quantization 
process. 

5. The method recited in claim 3, wherein the encoding 
comprises compressing the data with a discrete cosine 
transform process. 45 

6. The method recited in claim 1, further comprising the 
steps of: encoding the data with the encoder and providing 
the encoded data to the encoder buffer, wherein the encoding 
comprises compressing the data with a vector quantization 
process; multiplexing the data from the encoder buffer with 50 
data from at least one other encoder buffer; and transmitting 

a multiplex comprising the multiplexed data over a data 
channel to the decoder buffer. 

7. The method recited in claim 1, further comprising the 
steps of: encoding the data with the encoder and providing 
the encoded data to the encoder buffer, wherein the encoding 55 
comprises compressing the data with a discrete cosine 
transform process; multiplexing the data from the encoder 
buffer with data from at least one other encoder buffer; and 
transmitting a multiplex comprising the multiplexed data 
over a data channel to the decoder buffer. 60 

8. An apparatus for operating a video compression system 
comprising at least one encoder providing data to an encoder 
buffer and a decoder taking data from a decoder buffer, 
comprising: 

(a) means for changing the rate at which the encoder 65 
provides data to the encoder buffer from a first rate R 
to a second rate R'; and 



(b) a control computer operatively coupled to said 
encoder and being programmed to control the rate at 
which the encoder provides data to the encoder buffer 
and to maintain an encoder buffer occupancy b c (t) such 
that: (1) prior to time T 0 -T^ max{0, RT rf -B rf }^b,(i) 
^min{B e , RT^}; and (2) after time T 0 , max{0, RT rf - 
Bj} ib^OSminfB,, RT rf }; and (3) during the time T 0 
-T^t^T 0 , max{0, RTo-Rt+R't+RT^RTo-B,}^ 
(t)^min{B e , RTo-Rt+R't+R'T^-RTo}; wherein t rep- 
resents time, T 0 represents the time at which the data 
rate changes from R to R 1 , B d represents the maximum 
capacity of the decoder buffer, B € represents the maxi- 
mum capacity of the encoder buffer, and T d represents 
the delay caused by the encoder and decoder buffers. 

9. The apparatus recited in claim 8, further comprising 
means for multiplexing the data from the encoder buffer with 
data from at least one other encoder buffer, and transmitting 
a multiplex comprising the multiplexed data over a data 
channel to the decoder buffer. 

10. The apparatus recited in claim 8, further comprising 
means for compressing the data before providing the data to 
the encoder buffer. 

11. The apparatus recited in claim 8, further comprising: 
means for compressing the data with one of a vector 
quantization process or a discrete cosine transform process; 
means for multiplexing the data from the encoder buffer with 
data from at least one other encoder buffer; and means for 
transmitting a multiplex comprising the multiplexed data 
over a data channel to the decoder buffer, encoder provides 
data to its buffer is changed from a first rate R to a second 
rate R\ 

12. A system for providing a plurality of programs to a 
cable operator or subscriber having means for selecting a 
desired program for viewing, a decoder and a decoder buffer 
providing the selected program to the decoder, comprising: 

(a) first and second encoders each receiving program data; 

(b) first and second encoder buffers respectively coupled 
to said first and second encoders; 

(c) a multiplexor receiving data from said first and second 
encoders and outputting a multiplex to a data channel 
for reception by said cable operator or subscriber; and 

(d) a control computer coupled to said first and second 
encoders and to said multiplexor, comprising program 
means for changing the rates at which the encoders 
provide data to their encoder buffers and the encoder 
buffers provide data to the multiplexor, and for con- 
trolling the occupancies of the encoder buffers such that 
the decoder buffer neither overflows nor underflows 
when the rate at which either encoder provides data to 
its buffer is changed from a first rale R to a second rate 
R'; 

wherein said control computer maintains each encoder 
buffer occupancy b^O such that: (1) prior to time 
T 0 -T d , max{0, RT^B rf }<b.(t)<min{B e , RTJ; and (2) 
after time T 0 , max{0, R'T^Bj<b tf (t)<min{B„ RT d }; 
and (3) during the time T 0 -T d <l<T 0 , max{0, RT 0 - 
Rt+R , t+RT rf -R , T o -B rf }<b <r (0<min{B, f , RT 0 -Rt+R't+ 
RT rf -RT 0 }; wherein t represents time, T 0 represents 
the time at which the data rate changes from R to R', B d 
represents the maximum capacity of the decoder buffer, 
B, represents the maximum capacity of the encoder 
buffer, and T d represents the delay caused by the 
encoder and decoder buffers. 
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